Dynamic slabbing to render views of medical image data

ABSTRACT

Systems and methods for generating images of respective patients from multi-dimensional medical image data sets include a processor circuit in communication with a display having a graphic user interface (GUI), the processor circuit configured to define at least one grouping of a subset of slices selected from slices in a medical data stack, then generate and display at least one slabbed view based on the at least one grouping of slices.

RELATED APPLICATIONS

This application claims priority to U.S. Provisional Application Ser.No. 60/871,247, filed Dec. 21, 2006, the contents of which are herebyincorporated by reference as if recited in full herein.

FIELD OF THE INVENTION

The present invention relates to renderings of medical imaging data.

BACKGROUND OF THE INVENTION

Mammography is a medical imaging technique used to screen for breastcancer and other abnormalities in breast tissue. Traditionally,mammography images, referred to as mammograms, have been 2D images.Unlike the conventional mammogram, X-ray tomosynthesis is a mammographytechnique that creates a 3D representation of the breast. See, e.g.,US2006/0098855. A typical mode for viewing such a 3D model is bybrowsing a series of (parallel) 2D images, referred to as “a stack.” A2D image in a stack is also referred to as a slice.

The diagnostic viewing of digital mammography images is typicallyperformed in a Picture Archiving and Communication System (PACS). A maintask of the PACS is to provide a highly efficient workflow for thehealth professional to review the images. The desire for high diagnosticthroughput is particularly pertinent in the case of mammographyscreening, where an entire population of women can undergo mammographyimaging. Within a few seconds, the reading physician should be able tocome to the conclusion whether the breast is healthy or containssuspicious findings that should be further analyzed. When tomosynthesisimages are used, a whole stack of 2D images must be reviewed for eachbreast instead, in contrast to conventional review of just a single 2Dimage. The tomosynthesis technique may potentially multiply the requiredreview time by the number of images in the stack. Therefore, it is ofgreat interest to create navigation tools for tomosynthesis images thatcan increase the speed of the review process.

One conventionally important part of the diagnostic assessment of themammography images is to compare two breast images. To do so, the rightand left breast are typically displayed simultaneously in a mirroredsetup to allow a relatively simple visual comparison. Typically, theradiologist looks for symmetry between the right and left breast.Asymmetry could suggest that the images should be reviewed closer. Acomparison of the new image(s) with older images is also highly usefuland is common practice if older images exist.

The ability to have automatic geometric position synchronization betweendifferent stacks is a known PACS function, in a Sectra® PACSproduct/system it is known as the “Localizer”. It is believed that thisfeature is limited to stacks that have a known geometric relation,typically several scans in between which the patient did not move, whichin the DICOM standard is denoted by the “Frame of reference” attribute.Geometric position synchronization of stacks without limitations is alsoa conventional Sectra® PACS function, but the user manually defines acommon reference point for the two data sets.

Computer Aided Detection (CAD) refers to computer-based image analysismethods that automatically identify suspected abnormalities, theirlocation and other characteristics. CAD has proven to be useful formammography screening, primarily as a complement to manual review. Atypical work flow is that the mammograms undergo CAD analysis beforemanual review, where the CAD algorithm creates marks that are accessibleby the physician during the manual review.

Despite the above, there is a need to provide systems and methods thatcan generate views that improve the accuracy, speed and/or quality ofthe diagnostic assessment.

SUMMARY OF EMBODIMENTS OF THE INVENTION

Embodiments of the present invention are directed to methods, systemsand computer program products that can facilitate diagnostic reviews ofmedical images. Embodiments of the present invention may be particularlysuitable for analysis of tomosynthesis mammograms.

Embodiments of the invention are particularly suitable for PACS, theinvention is, however, more generally applicable, and should not beinterpreted to be limited to PACS.

Some embodiments are directed to interactive visualization systems forrendering images of respective patients from a multi-dimensional medicalimage data set.

Embodiments of the invention are directed to interactive visualizationsystems for generating images of respective patients frommulti-dimensional medical image data sets. The systems include a signalprocessor circuit in communication with a display having a graphic userinterface (GUI), the processor circuit configured to define at least onegrouping of a subset of slices selected from slices in a medical datastack, then generate and display at least one view of a part of the dataset within a volumetric region defined by the at least one grouping ofslices.

The generated and displayed view can be a slabbed view based on the atleast one grouping of slices.

The medical image data set can include a tomosynthesis mammography dataset.

In some embodiments, the processor circuit is configured to employComputer Aided Detection (CAD) to electronically analyze and identifysuspected abnormalities in slices of the medical image data stack, andwherein the circuit is configured to define a plurality of initialgroupings of different stack slices based on data from the CADidentification.

The system can be configured so that the number of slices in arespective initial grouping of slices is less where there is at leastone CAD identified suspected abnormality relative to the number ofslices in initial slice groupings that do have not have at least one CADidentified suspected abnormality.

The system can be configured so that the GUI allows a user to refine aslabbed view whereby the current group of slices associated with thatslabbed view is split into a plurality of separate smaller groups ofslices and/or iteratively into a single slice view.

In particular embodiments, the circuit can be configured to generate agrouping of slices using CAD to identify clusters of microcalcificationssuch that each cluster is substantially entirely within a singlegrouping.

In some embodiments, the circuit is configured to provide an electronicreview wizard with an automated viewing protocol that allows a user tobrowse slabbed views.

Other embodiments are directed to methods of generating views of medicaldata. The methods include: (a) electronically generating groups ofslices from a stack of slices; (b) combining pixel values associatedwith a common position in each slice of a respective group; and (c)generating slabbed views, one for each of the respective grouped slices,based on the combined pixel values.

The methods can also include refining at least one of the slabbed viewsinto different sub-groupings of the original group of slices for therefined slabbed view.

Still other embodiments are directed to signal processor circuits forgenerating medical diagnostic views of a tomosynthesis stack ofmammography patient image data. The circuit is configured to: (a)communicate with a graphical user interface (GUI) associated with aclient workstation to accept user input to interact with the image datato generate desired views of the image data, and (b) generate and allowa user to browse slabbed views of breast tissue.

Yet other embodiments are directed to computer program products forproviding a physician interactive access to patient medical data forrendering diagnostic medical images. The computer program productincludes a computer readable storage medium having computer readableprogram code embodied in the medium. The computer-readable program codeincludes computer readable program code configured to define groups ofslices of a medical image data stack together; and computer readableprogram code configured to generate a respective slabbed view for eachof the groups of slices.

It is noted that any of the features claimed with respect to one type ofclaim, such as a system, apparatus, or computer program, may be claimedor carried out as any of the other types of claimed operations orfeatures.

Further features, advantages and details of the present invention willbe appreciated by those of ordinary skill in the art from a reading ofthe figures and the detailed description of the preferred embodimentsthat follow, such description being merely illustrative of the presentinvention.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a schematic diagram of an electronic visualization system thatcan be used to render and display (medical) images

FIG. 2 is a schematic illustration of an imaging visualization systemsuch as a PACS according to embodiments of the present invention.

FIGS. 3A and 3B are schematic illustrations of two tomosynthesis stacksof patient image slices. FIG. 3A illustrates a geometric reference pointidentified in each of the stacks. FIG. 3B illustrates that theanatomical regions in the two data sets can be electronically connected,correlated or linked according to embodiments of the present invention.

FIG. 4A illustrates a 2D reference image with salient image features.

FIG. 4B illustrates identification of features corresponding to thoseshown in FIG. 4A, in slices of a tomosynthesis stack, with theidentified slices being electronically slabbed to generate a viewaccording to embodiments of the present invention.

FIG. 5 is a schematic illustration of matching anatomical extent to arange in a Z dimension to slices in each tomosynthesis stack accordingto embodiments of the present invention.

FIG. 6 illustrates mirroring two views about a vertical axis for visualcomparison according to embodiments of the invention.

FIG. 7A illustrates a display with a visual tomo-synch mode alertaccording to embodiments of the invention.

FIG. 7B illustrates a display with a visual tomo-synch mode alert and auser interface (typically a GUI) to allow a user to activate or reapplya tomo-synch mode.

FIG. 8 is a schematic illustration of a slabbing technique according toembodiments of the present invention.

FIGS. 9A-9D illustrate a series of operations that can be used to groupslices in tomosynthesis stacks, view slices and slabs, and refine viewsto different sub-groups and/or original slice views according toembodiments of the present invention.

FIGS. 10A and 10B are schematic illustrations of a main view of atomosynthesis data set with a list of identified CAD marks for ease ofuser review according to embodiments of the present invention.

FIGS. 11A-11C illustrate rendered diagnostic views of patient image datausing vicinity marks according to embodiments of the present invention.

FIG. 12 is a block diagram of a data processing circuit or systemaccording to embodiments of the present invention.

DETAILED DESCRIPTION OF EMBODIMENTS OF THE INVENTION

The present invention now is described more fully hereinafter withreference to the accompanying drawings, in which embodiments of theinvention are shown. This invention may, however, be embodied in manydifferent forms and should not be construed as limited to theembodiments set forth herein; rather, these embodiments are provided sothat this disclosure will be thorough and complete, and will fullyconvey the scope of the invention to those skilled in the art.

Like numbers refer to like elements throughout. In the figures, thethickness of certain lines, layers, components, elements or features maybe exaggerated for clarity. Broken lines illustrate optional features oroperations unless specified otherwise. In the claims, the claimedmethods are not limited to the order of any steps recited unless sostated thereat.

The terminology used herein is for the purpose of describing particularembodiments only and is not intended to be limiting of the invention. Asused herein, the singular forms “a”, “an” and “the” are intended toinclude the plural forms as well, unless the context clearly indicatesotherwise. It will be further understood that the terms “comprises”and/or “comprising,” when used in this specification, specify thepresence of stated features, integers, steps, operations, elements,and/or components, but do not preclude the presence or addition of oneor more other features, integers, steps, operations, elements,components, and/or groups thereof. As used herein, the term “and/or”includes any and all combinations of one or more of the associatedlisted items. As used herein, phrases such as “between X and Y” and“between about X and Y” should be interpreted to include X and Y. Asused herein, phrases such as “between about X and Y” mean “between aboutX and about Y.” As used herein, phrases such as “from about X to Y” mean“from about X to about Y.”

Unless otherwise defined, all terms (including technical and scientificterms) used herein have the same meaning as commonly understood by oneof ordinary skill in the art to which this invention belongs. It will befurther understood that terms, such as those defined in commonly useddictionaries, should be interpreted as having a meaning that isconsistent with their meaning in the context of the specification andrelevant art and should not be interpreted in an idealized or overlyformal sense unless expressly so defined herein. Well-known functions orconstructions may not be described in detail for brevity and/or clarity.

It will be understood that, although the terms first, second, etc. maybe used herein to describe various elements, components, regions, layersand/or sections, these elements, components, regions, layers and/orsections should not be limited by these terms. These terms are only usedto distinguish one element, component, region, layer or section fromanother region, layer or section. Thus, a first element, component,region, layer or section discussed below could be termed a secondelement, component, region, layer or section without departing from theteachings of the present invention. The sequence of operations (orsteps) is not limited to the order presented in the claims or figuresunless specifically indicated otherwise.

“Navigation” refers to electronically moving between different views ofthe data set. The most straight-forward navigation is to switch whichslice is shown by the main display, but it may also mean reconstructingother representative images from the data set, such as slabbing a numberof slices, constructing maximum-value slabs in arbitrary direction(known as Maximum Intensity Projection or “MIP”), constructing 2D slicesin an arbitrary or direction different from the slices (known asMultiplanar Reconstruction, MPR), semi-transparent renderings of 3Dregions using Direct Volume Rendering, and other methods. Below, X and Ydimensions refer to the plane of a 2D image, whereas the Z dimensionrefers to the direction perpendicular to X and Y.

The term “Direct Volume Rendering” or DVR is well known to those ofskill in the art. DVR comprises electronically rendering a medical imagedirectly from data sets to thereby display visualizations of targetregions of the body, which can include color as well as internalstructures, using multi-dimensional 3D or 4D or more dimensional data.In contrast to conventional iso-surface graphic constructs, DVR does notrequire the use of intermediate graphic constructs (such as polygons ortriangles) to represent objects, surfaces and/or boundaries. However,DVR can use mathematical models to classify certain structures and canuse graphic constructs.

Also, although embodiments of the present invention are directed toX-ray tomosynthesis, other 3-D image generation techniques and otherimage data may also be used.

The term “automatically” means that the operation can be substantially,and typically entirely, carried out without human or manual input, andis typically programmatically (via computer program control) directed orcarried out. The term “electronically” includes both wireless and wiredconnections between components.

The term “synchronized” and derivatives thereof means that the sameoperation is applied to two or more views, generally, if notsubstantially or totally, concurrently. Synchronization is differentfrom registration, where two volumes are merely aligned. Thesynchronization operation can be carried out between at least twodifferent sets of image data, where an operation on a view rendered froma first data set is automatically synchronized (applied) to the sameview as rendered from a different second image data set. It is notedthat there can be any number of views in a synch group. Further, thesynchronization does not require a static “master-slave” relationshipbetween the images. For example, particularly, where two tomosynthesisdata sets are synched, if an operation on image 1 is synched to image 2,then an operation on image 2 can also be synched to image 1 as well. Inaddition, in some embodiments, there can be several synch groupsdefined, and the synch operation can be applied across all groups,between defined groups, or within a single group, at the same time.

The term “slabbing” and derivatives thereof refer to a merging of two ormore image slices in a stack and may, for example, use the maximum,minimum, median or average slice pixel value or other valuation of acombination or accumulation of the individual pixel values in the synchslices included in a respective slab.

The term “Computer Aided Detection (CAD)” refers to computer-based imageanalysis methods that automatically identify suspected abnormalities,their location and possibly other visual or anatomical characteristics.

The term “clinician” means physician, radiologist, physicist, or othermedical personnel desiring to review medical data of a patient. The term“tissue” means anatomical entities such as organs, blood vessels, boneand the like.

Visualization means to present medical images to a user/clinician forviewing. The visualization can be in a flat 2-D and/or in 2-D whatappears to be 3-D images on a display, data representing features withdifferent visual characteristics such as with differing intensity,opacity, color, texture and the like. The images as presented by thevisualization do not have to be the same as the original construct(i.e., they do not have to be the same 2-D slices from the imagingmodality). Two common visualization techniques (apart from viewingoriginal slices) are Multiplanar Reconstruction (MPR), which shows anarbitrary oblique slice through the anatomy and Maximum IntensityProjection (MIP) where a slab is visualized by displaying the maximumvalue “seen” from each image pixel. For MPR, there are a number ofvariants, the slice can be thin or constructed by averaging a thickerslab, etc . . . .

A data set can be defined as a number of grid points in G dimensions,where there is V number of values in each grid point. The term“multi-dimensional” refers to both components, grid G and variates V, ofthe data sets. For data sets having a V≧1, the data set is referred toas multi-variate. For example, normal medical data sets have G=3 andV=1.

The term “primary” refers to a data set or images or views generatedbased thereon, that is different from the reference data set, and istypically the more recent (or current) data set or the data set with apotential tissue irregularity.

The term “vicinity mark(s)” refers to an electronically generated markthat represents a feature, abnormality or irregularity associated with adifferent view or a different slice from the current view, that wouldnot normally be shown or appear in the current view. As such, thevicinity mark can represent a feature in close spatial proximity orrelationship to visually alert the user/clinician that a relevantfeature may be in a different view.

In the description that follows, a client-server setup is illustrated,but the data retrieval interfaces contemplated by the instant inventionmay be implemented within one computer as well. The term “client” willbe used both to denote a computer and the software (application) runningon the computer. Additional computers can be used including more thanone server and/or more than one client for a workstation. For example,the server can be more than one server with different functions carriedout by or between different servers, such as the patient data short orlong-term storage can be on one or more separate servers. The terms“display circuit” and/or “processor circuit” refer to software and/orhardware components that generate a view of image data for display. Thecircuits may be implemented using a variety of hardware and software.For example, operations of the display and/or processor circuit may beimplemented using special-purpose hardware, such as an ApplicationSpecific Integrated Circuit (ASIC) and programmable logic devices suchas gate arrays, and/or software or firmware running on a computingdevice such as a microprocessor, microcontroller or digital signalprocessor (DSP). The display and/or processor circuit is not limited toa graphics card or similar hardware and portions of the circuit mayreside on different components of the visualization system.

Turning now to FIG. 1, an exemplary visualization system 10 isillustrated. As known to those of skill in the art, the system 10 caninclude at least one server 20 s with an image import module 15, patientdata storage 20, a data fetch module 21, a client (and/or workstation)30 and a rendering system 25. The visualization system 10 can be incommunication with at least one imaging modality 11 that electronicallyobtains respective volume data sets of patients and can electronicallytransfer the data sets to the electronic storage 20. The imagingmodality 11 can be any desirable modality such as, but not limited to,NMR, MRI, ultrasound, and X-ray of any type, including, for example,tomosynthesis. Computed Tomography (CT) and fluoroscopy. Typically, formammograms, the modality is of X-ray type. The visualization system 10may also operate to render images using data sets from more than one ofthese modalities. That is, the visualization system 10 may be configuredto render images irrespective of the imaging modality data type (i.e., acommon system may render images for both CT and MRI volume image data).In some embodiments, the system 10 may optionally combine image datasets generated from different imaging modalities 11 to generate acombination image for a patient.

The rendering system 25 can be in communication with a physicianworkstation 30 to allow user input (typically graphical user input(“GUI”)) and interactive collaboration of image rendering to give thephysician alternate image views of the desired features in generally,typically substantially, real time. The rendering system 25 can beconfigured to zoom, rotate, and otherwise translate to give thephysician visualization of the patient data in one or more views, suchas section, front, back, top, bottom, and perspective views. Therendering system 25 may be wholly or partially incorporated into thephysician workstation 30, or can be a remote or local module (or acombination remote and local module) component or circuit that cancommunicate with a plurality of physician workstations (not shown). Thevisualization system can employ a computer network and may beparticularly suitable for clinical data exchange/transmission over anintranet. A respective workstation 30 can include at least one display31 (and may employ two or more adjacent displays). The workstation 30and/or rendering system 25 form part of an image processor system thatincludes a digital signal processor and other circuit components thatallow for collaborative interactive user input using the display at theworkstation 30. Thus, in operation, the image processor system rendersthe visualization of the medical image using the medical image volumedata, typically on at least one display at the physician workstation 30.

As shown in FIG. 2, each respective workstation 30 can be described as aclient 30 (shown as 30 a, 30 b, 30 c, . . . 30 e) that communicates withat least one (hub or remote) server 20 s that stores the patient datasets or is in communication with the stored patient electronic datafiles 20. Additional numbers of clients 30 may be in communication withthe server 20 s and more than one server 20 s may be used to storepatient data. A data retrieval interface 50 can be used to communicatewith the clients 30 a-30 e and the stored data sets on and/or accessiblevia server 20 s. Some of the clients, shown as clients 30 a, 30 b, 30 ccan be local (within a common clinic or facility) and can access thedata sets via a relatively broadband high speed connection using, forexample, a LAN, while others, shown as clients 30 d, 30 e, designated bythe broken line, may be remote and/or may have lesser bandwidth and/orspeed, and for example, may access the data sets via a WAN and/or theInternet. Firewalls may be provided as appropriate for security.

For ease of discussion, the data retrieval interface 50 is shown as astand-alone module or circuit. However, the interface 50 can be disposedpartially on each client 30, partially or wholly on the server 20 s, ormay be configured as a discrete data retrieval interface server 50 s(not shown). The clients 30, server 20 s and/or interface 50 can eachinclude a digital signal processor, circuit and/or module that can carryout aspects of the present invention. As shown in FIG. 2, all orselected ones of the clients 30 a-30 e can be online at the same timeand may each repeatedly communicate with the data retrieval interface 50to request volume image data, potentially resulting in a speed penaltyand inhibiting fast system performance.

Embodiments of the invention are directed to visualization systems andmethods that can define and display a view of the primary tomosynthesisstack, based on properties of the reference data set itself and/orproperties of the current view of the reference data set. FIG. 3Aillustrates a primary data set 200 that is a tomosynthesis stack and areference data set 100 that is another image or stack of images for abreast. In the embodiment shown in FIGS. 3A and 3B, the reference dataset 100 is also a tomosynthesis data set loot. In the embodiment shownin FIG. 4, the reference data set 100 can be a 2D reference image, suchas a conventional mammogram. Views from both data sets can be shown indifferent parts of the same viewing application on a single display oron different displays. In some embodiments, the visualization system 10can be configured to display a conventional reference mammogram, andcorresponding views from a plurality of subsequent tomosynthesis stacksfor comparing changes in density and symmetry, over time.

Embodiments of the invention can automatically define and display a viewof the primary tomosynthesis stack, based on properties of the referencedata set itself and/or properties of the current view of the referencedata set. Such properties can be slice order number, knowngeometric/anatomic position, manually defined reference pointcorresponding to a point in the primary data set, but also results froman analysis of image content of the primary and reference data setswhere comparable features can be identified in order to select the bestview.

FIGS. 3A and 3B schematically illustrate a geometric/anatomic positionin the case of two tomosynthesis stacks. FIG. 3A shows that a geometric(positional) reference point is identified (automatically or manually),connecting or linking 130 corresponding anatomical regions 125 in thetwo data sets 100, 200. FIG. 3B illustrates that navigation 150 in onestack 100 t, 200 t causes the corresponding navigation in the otherstack. As shown in FIGS. 3A and 3B, the number of slices in each stack100 t, 200 t does not need to be the same as the synchronization isconnected to patient anatomy/data set geometry.

FIGS. 4A and 4B illustrate another embodiment, where the reference imagedata set 100 is a 2D mammogram image, a relatively common case whencomparing a new tomosynthesis stack 200 with a conventional 2D mammogram(a prior). It is contemplated that a particularly useful application ofsome embodiments of the invention can be the ability to find a view ofthe primary tomosynthesis data set that well corresponds to the 2Dmammogram, so as to present the corresponding anatomical region in asimilar way. The view can be a certain slice selected from the stack, aplurality of discrete slices, a slabbed view of a set of slices or anMPR slice in another direction.

FIGS. 4A and 4B illustrate that embodiments of the invention can find atomosynthesis view corresponding to the 2D reference image 100 m. FIG.4A shows a 2D reference image 100 m with two salient image features 101,102.

FIG. 4B illustrates the corresponding features are identified in twoslices 220 s ₁, 220 s ₂ of the primary tomosynthesis stack 200. Asshown, the best corresponding view is identified as a maximum value-slab225 of the two slices 220 s ₁, 220 s ₂.

As described above with respect to FIGS. 3A and 3B, in otherembodiments, the reference data set 100 is also a tomosynthesis stack100 t. This can occur in a new image

prior image situation, or a left breast

right breast comparison within a new examination. The arrows representwork flow or navigation between views. In this case, the system can beconfigured to create a view of the primary data set 200 thatsubstantially corresponds to the current view of the reference data set100, for instance where the anatomical extent of the reference view ismatched in the primary view. FIG. 5 illustrates that each stack 100 t,200 t may include different numbers of slices. Each slice in thereference data set 121 s covers a larger anatomical extent 110 in the Zdirection relative to the slices 221 s in the primary data stack 200 tand their smaller anatomical extents 210.

FIG. 5 schematically illustrates an example of matching anatomicalextent. One view 121 v is selected from the reference stack 100 t,corresponding to a range in the Z dimension. The synchronized primaryview 221 v to the right automatically finds the best match of the Zdimension extent, in this case a slab 225 of three slices 221 s.

In some embodiments, the system 10 can be configured so that the primarydata set 200 can be rotated and flipped according to the reference dataset 100, for instance to create a mirror image setup for easy visualcomparison. Mirroring can be applied both to new and old views of thesame breast and to views of the left and right breast. FIG. 6illustrates mirroring two views across a vertical axis for simplecomparison.

In some embodiments, the system can be configured so that interactionwith either of the data sets 100, 200 can be configured to create acorresponding change of the other data set automatically. For example,browsing to another slice in the primary data set 200 can cause aresynchronization of the reference data set 100 at the new slice'sgeometric position. In other words, the synchronization is not requiredto be a one-time occurrence but rather a mode of viewing that is activeuntil explicitly interrupted. Other viewing settings can also beconnected to synchronization, such as grayscale window/level setting,zoom, rotation, flip, panning, etc.

To alert a user as whether the tailored comparison view is active or notbetween the two data sets, an alert (audio and/or visual feedback) canbe generated/displayed. For example, visual indicia viewable through aGUI, such as displaying an active icon or symbol in color, or displayinga header or footer with the active comparison mode visually indicated as“on” or “off”. Audio output may also be used as appropriate(particularly after initial activation by a user, the mode isautomatically disengaged because of the user's browsing of the primarytomosynthesis data set when the reference data set is a 2D image). FIG.7A illustrates a display 30 showing a reference view 100 v and a primaryview 200 v with the automated comparison viewing mode inactivated (TomoSynch mode is off), while FIG. 7B illustrates the viewing mode activatedas indicated by visual indicator 450 (shown as lit graphic “Tomo Synch”noting the active mode) in FIG. 7B.

One example is that a first comparable view of a tomosynthesis stack 200t and a 2D mammogram 100 m is created, then the user browses thetomosynthesis stack 200 t. This means that the electronic correlation orconnection between the data sets may be broken, which can be indicatedto the user. On the other hand, if the reference data set is anothertomosynthesis stack 100 t, then the corresponding browsing or viewregeneration may be done there as well and the comparability can remainintact.

It is contemplated that a desired viewing format will be the traditionalway of browsing through a stack of slices, but the efficiency may beincreased by reducing the number of slices as much as practicable. Thus,slabbing or otherwise grouping slices (of oblique views or neighborhoodbased analysis) can be useful, if applied in a way to preservediagnostically important features. A subset of slices from a data setcan be selected and grouped, then a corresponding image can be generatedand displayed of that part of the data set within a volumetric regiondefined by the at least one grouping of slices.

Slabbing can be defined mathematically as follows. A group of n 2D sliceimages defines a set, this set is denoted by I. For an arbitrary pixelposition (x,y), mutual for all slices, the corresponding value of eachslice is defined as v_(i). Thus, for the entire group, an array can bedefined as:v=(v₁,v₂, . . . , v_(n)).  Equation 1The pixel value for the joint view of the whole group is constructed bya function ƒ(v), applied individually to all pixel positions of theslices. Average value slabbing is achieved by:

$\begin{matrix}{{f\left( \overset{\_}{v} \right)} = {\sum\limits_{i = 1}^{n}{v_{i}/{n.}}}} & {{Equation}\mspace{14mu} 2}\end{matrix}$Maximum value slabbing is achieved by:

$\begin{matrix}{{f\left( \overset{\_}{v} \right)} = {\max\limits_{i \in l}{\left( v_{i} \right).}}} & {{Equation}\mspace{14mu} 3}\end{matrix}$

FIG. 8 illustrates a group of slices 221 g having a pixel position 221 pidentified and/or marked and used to render a slabbed view 225. In theslabbed view 225, each pixel 225 p is a combination of the pixel valuesat the corresponding pixel position of the slices in the group 221 g. Acommon size of a 2D mammogram is in the range of millions of pixels,typically about 3328×2560 pixels (8.5 million pixels).

In some embodiments, data dependent slabbing can be used for efficientanalysis, rendering and/or reading of medical image stacks. An exemplaryworkflow would be as shown schematically in FIGS. 9A-9D.

-   -   1. As shown in FIG. 9A, an original medical image data set, the        stack, is electronically automatically analyzed for relevant or        visually dominant features. For example, tissue intensity        variation, dimpling, density, calcifications (e.g.,        microcalcifications and/or clusters thereof), abrupt transitions        from neighborhoods of pixels, shapes or other optically or        electronically detectable visual irregularities, and the like.    -   2. As shown in FIG. 9B, an initial grouping 200 g ₁ of stack        slices 221 is selected based on the result of the analysis. A        plurality of groups of slices 221 g are created. The group size        (number or slices per group and number of groups) can vary.        Typically, relatively small groups 221 g are created when there        are important features in the slices, and larger groups (larger        numbers of slices) for less important features.    -   3. FIG. 9C illustrates an initial set of views are generated 200        v ₁, one for each slice group 221 g. The view 200 v 1 can        include slabbed versions 225 of the grouped slices in the group.        The slabbing can be constructed by various compositing        algorithms, such as averaging, maximum value, minimum value, or        others.    -   4. The user can view the data set by browsing the initial view        200 v 1 with slab views 225.    -   5. FIG. 9D illustrates that a user may find an interesting        slabbed view and decide to analyze it in more detail. This can        be done by activating a refine mechanism (typically by pressing        a key or clicking a button in the GUI), which splits the current        slabbed group 225 group into several groups and/or into the        original slices. The refinement can repeated until the slab        views are undone to reveal the original slices (see top slab 225        in FIG. 9D which is back to the original slices 221 s from the        grouping in FIG. 9B).        The refinement can be applied to a single group or the whole        stack at once. The refinement can go directly to original        slices, or have intermediate steps. The groups may be        overlapping.

The slice groups can be determined based on the outcome of a CADalgorithm. One usage is to have a maximum size of a slice groupcontaining CAD findings. Another possibility is to make sure thatsimilar and nearby CAD findings end up in the same slice group, asituation that may apply for clusters of micro-calcifications.

The described work flow can also be implemented as an automatic viewingprotocol, which can be referred to as a “review wizard” that has aseries of logical rules that define a list of view combinations andtheir layout. This can be an extension of the regular “hanging”protocols for PACSs, known as Default Display Protocols (DDPs). Toreview the whole data set, the user simply needs to press a key or clicka button for each step through the wizard. The wizard can be defined toshow the initial slice groups. The wizard can be defined toautomatically show certain refinements. Apart from controlled browsingof slabbed stacks, the wizard may define other view sequences for thedata set, including zoomed and panned views, different grayscalewindows, with and without image processing algorithms applied, etc.

In some embodiments, the systems can implement an efficient work flowfor diagnostic review of medical image stacks or volumes using CADmarks, where CAD marks are available to let the user navigate betweenviews of the data set by means of location information from CAD marks.

CAD marks can be used in a number of ways to achieve desired navigation.In some embodiments, a viewing mode can include a simple or summary listof the CAD marks can be displayed adjacent to the main image view,preferably sorted according to location. The list entry may contain someof the available information about the mark. When the user selects alist entry, typically by clicking it, a representative viewcorresponding to this mark is shown on the display, typically in themain view. A variant of this mode is to list or show small images,thumbnails, for each mark or mark cluster, where the thumbnail shows aminiature of what the main view will show when this thumbnail isselected. To navigate to this view, the user selects (clicks) thethumbnail. The currently selected thumbnail can be visually marked inthe list.

FIGS. 10A and 10B illustrate a main view 100 m, 200 m of a reference orprimary data set 100, 200. FIG. 10A also shows the use of a list ofuser-selectable marks that allow a user to generate the view(s)associated with the CAD mark selected. FIG. 10B illustrates a list ofCAD mark thumbnails 500 t. Combinations of the visual thumbnails andlist 500 t, 500 can also be used.

Other modes consist of (direct) interaction with the main view. Toinform the user that there are CAD marks that do not belong in thecurrently shown view, but are located in the vicinity, such marks can beshown with a slightly different appearance relative to a mark “inside”the view. FIGS. 11A-11C, illustrate the use of vicinity marks 600. Thevicinity marks 600 could, for instance, be shown in contrast, grayed outand/or have dashed or broken lines. The vicinity mark appearance couldalso depend on the distance from the current view, for increasingdistance the marks could become fainter in color or smaller in size (asshown by the fainter color and smaller size from left to right in FIGS.11A-11C). One example is a ring marker that could be modeled as sphereor ellipsoid, showing up as smaller circles in nearby views. Theappearance can be made to differ if the exact position is “below orabove” the current view (so that the user intuitively knows whetherhe/she should browse forward or backward to get to it). Regardless ofthe appearance, navigation can be achieved by the user selecting avicinity mark, to advance toward, and typically directly switching tothe view where the mark most properly belongs.

The CAD findings and/or marks can also be used as a base for anautomatic viewing protocol, referred to as a review wizard that definesa list of view combinations and their layout. This can be seen as anextension of the regular “hanging” protocols for PACSs known as DefaultDisplay Protocols (DDPs). To review the whole data set, the user simplyneeds to press a key or click a button for each step through the wizard.The wizard can be defined to visit CAD marks in a certain order, forinstance sorted by location or probability of abnormality. Apart fromcontrolled browsing of CAD mark related views, the wizard may defineother view sequences for the data set, including zoomed and pannedviews, different grayscale windows, with and without image processingalgorithms applied, etc.

The CAD findings and/or marks used for navigation can be automaticallyclustered in relevant groups. For example, one clustering is to join allmarks corresponding to the same slice in a group. There can also beseparate marks that correspond to what a physician would refer to as a“single” feature, such as a cluster of microcalcifications. Joiningmarks from several slices into a group can be combined with the creationof a representative view to cover the entire region of interest definedby the marks from the several slices. Typical examples include slabbing(combining several slices into a single slice, typically by averaging)and 3D modeling such as DVR.

As will be appreciated by one of skill in the art, embodiments of theinvention may be embodied as a method, system, data processing system,or computer program product. Accordingly, the present invention may takethe form of an entirely software embodiment or an embodiment combiningsoftware and hardware aspects, all generally referred to herein as a“circuit” or “module.” Furthermore, the present invention may take theform of a computer program product on a computer-usable storage mediumhaving computer-usable program code embodied in the medium. Any suitablecomputer readable medium may be utilized including hard disks, CD-ROMs,optical storage devices, a transmission media such as those supportingthe Internet or an intranet, or magnetic or other electronic storagedevices.

Computer program code for carrying out operations of the presentinvention may be written in an object oriented programming language suchas Java, Smalltalk or C++. However, the computer program code forcarrying out operations of the present invention may also be written inconventional procedural programming languages, such as the “C”programming language or in a visually oriented programming environment,such as VisualBasic.

Certain of the program code may execute entirely on one or more of theuser's computer, partly on the user's computer, as a stand-alonesoftware package, partly on the user's computer and partly on a remotecomputer or entirely on the remote computer. In the latter scenario, theremote computer may be connected to the user's computer through a localarea network (LAN) or a wide area network (WAN), or the connection maybe made to an external computer (for example, through the Internet usingan Internet Service Provider). In some embodiments, some program codemay execute on local computers and some program code may execute on oneor more local and/or remote server. The communication can be done inreal time or near real time or off-line using a volume data set providedfrom the imaging modality.

The invention is described in part below with reference to flowchartillustrations and/or block diagrams of methods, systems, computerprogram products and data and/or system architecture structuresaccording to embodiments of the invention. It will be understood thateach block of the illustrations, and/or combinations of blocks, can beimplemented by computer program instructions. These computer programinstructions may be provided to a processor of a general-purposecomputer, special purpose computer, or other programmable dataprocessing apparatus to produce a machine, such that the instructions,which execute via the processor of the computer or other programmabledata processing apparatus, create means for implementing thefunctions/acts specified in the block or blocks.

These computer program instructions may also be stored in acomputer-readable memory or storage that can direct a computer or otherprogrammable data processing apparatus to function in a particularmanner, such that the instructions stored in the computer-readablememory or storage produce an article of manufacture includinginstruction means which implement the function/act specified in theblock or blocks.

The computer program instructions may also be loaded onto a computer orother programmable data processing apparatus to cause a series ofoperational steps to be performed on the computer or other programmableapparatus to produce a computer implemented process such that theinstructions which execute on the computer or other programmableapparatus provide steps for implementing the functions/acts specified inthe block or blocks.

As illustrated in FIG. 12, embodiments of the invention may beconfigured as a data processing system 316, which can be used to carryout or direct operations of the rendering, and can include a processorcircuit 300, a memory 336 and input/output circuits 346. The dataprocessing system may be incorporated in, for example, one or more of apersonal computer, workstation 316, server, router or the like. Thesystem 316 can reside on one machine or between a plurality of machines.The processor 300 communicates with the memory 336 via an address/databus 348 and communicates with the input/output circuits 346 via anaddress/data bus 349. The input/output circuits 346 can be used totransfer information between the memory (memory and/or storage media)336 and another computer system or a network using, for example, anInternet protocol (IP) connection. These components may be conventionalcomponents such as those used in many conventional data processingsystems, which may be configured to operate as described herein.

In particular, the processor 300 can be commercially available or custommicroprocessor, microcontroller, digital signal processor or the like.The memory 336 may include any memory devices and/or storage mediacontaining the software and data used to implement the functionalitycircuits or modules used in accordance with embodiments of the presentinvention. The memory 336 can include, but is not limited to, thefollowing types of devices: ROM, PROM, EPROM, EEPROM, flash memory,SRAM, DRAM and magnetic disk. In some embodiments of the presentinvention, the memory 336 may be a content addressable memory (CAM).

As further illustrated in FIG. 12, the memory (and/or storage media) 336may include several categories of software and data used in the dataprocessing system: an operating system 352; application programs 354;input/output device drivers 358; and data 356. As will be appreciated bythose of skill in the art, the operating system 352 may be any operatingsystem suitable for use with a data processing system, such as IBM®,OS/2®, AIX® or zOS® operating systems or Microsoft® Windows®95,Windows98, Windows2000 or WindowsXP operating systems Unix or Linux™.IBM, OS/2, AIX and zOS are trademarks of International Business MachinesCorporation in the United States, other countries, or both while Linuxis a trademark of Linus Torvalds in the United States, other countries,or both. Microsoft and Windows are trademarks of Microsoft Corporationin the United States, other countries, or both. The input/output devicedrivers 358 typically include software routines accessed through theoperating system 352 by the application programs 354 to communicate withdevices such as the input/output circuits 346 and certain memory 336components. The application programs 354 are illustrative of theprograms that implement the various features of the circuits and modulesaccording to some embodiments of the present invention. Finally, thedata 356 represents the static and dynamic data used by the applicationprograms 354 the operating system 352 the input/output device drivers358 and other software programs that may reside in the memory 336.

The data 356 may include (archived or stored) multi-dimensional patientdigital image data sets 326 that provides at least one stack oftomosynthesis image data correlated to respective patients. As furtherillustrated in FIG. 12, according to some embodiments of the presentinvention application programs 354 include a Slabbing Module 325. Theapplication program 354 may be located in a local server (or processor)and/or database or a remote server (or processor) and/or database, orcombinations of local and remote databases and/or servers.

While the present invention is illustrated with reference to theapplication programs 354 in FIG. 12, as will be appreciated by those ofskill in the art, other configurations fall within the scope of thepresent invention. For example, rather than being application programs354 these circuits and modules may also be incorporated into theoperating system 352 or other such logical division of the dataprocessing system. Furthermore, while the application program 354 isillustrated in a single data processing system, as will be appreciatedby those of skill in the art, such functionality may be distributedacross one or more data processing systems in, for example, the type ofclient/server arrangement described above. Thus, the present inventionshould not be construed as limited to the configurations illustrated inFIG. 12 but may be provided by other arrangements and/or divisions offunctions between data processing systems. For example, although FIG. 12is illustrated as having various circuits and modules, one or more ofthese circuits or modules may be combined or separated without departingfrom the scope of the present invention.

The circuit can be configured so that when displaying a tomosynthesisstack and a reference data set, to automatically generate a view of thetomosynthesis stack tailored for comparison with the current view of thereference data set. The automated analysis can use data set featuresand, if a geometric relation between the data sets is known, positionlandmarks, displayed in different parts of the same viewing application.The circuit can be configured, at user interaction with the primarytomosynthesis stack, to automatically update the view of the referencestack (the synchronization can be in the other direction as well). Thecircuit can be configured to generate a view of a slabbing of severalslices, for instance a maximum or average slab. The view generated mayinclude image manipulations such as rotating, flipping, panning,zooming, grayscale window/level setting. The automatic view generationcan be a synchronization mode that is reapplied as the user interactswith the data sets. The circuit can be configured to display a visualmark to the user that states whether the tailored comparison view synchis active.

The foregoing is illustrative of the present invention and is not to beconstrued as limiting thereof. Although a few exemplary embodiments ofthis invention have been described, those skilled in the art willreadily appreciate that many modifications are possible in the exemplaryembodiments without materially departing from the novel teachings andadvantages of this invention. Accordingly, all such modifications areintended to be included within the scope of this invention as defined inthe claims. The invention is defined by the following claims, withequivalents of the claims to be included therein.

1. An interactive visualization system for generating images ofrespective patients from multi-dimensional medical image data sets,comprising: a processor circuit in communication with a display having agraphic user interface (GUI), the processor circuit configured to employComputer Aided Detection (CAD) to electronically analyze and identifysuspected abnormalities in slices of the medical image data stack toautomatically define at least one grouping of a subset of slicesselected from slices in a medical data stack based on data from the CADidentification, then generate and display at least one slabbed view of apart of the data set within a volumetric region defined by the at leastone grouping of slices with suspected abnormalities.
 2. A systemaccording to claim 1, wherein the medical image data set comprises atomosynthesis mammography data set.
 3. A system according to claim 2,wherein the circuit is configured to generate a grouping of slices usingCAD to identify clusters of microcalcifications such that each clusteris substantially entirely within a single grouping.
 4. A systemaccording to claim 1, wherein the circuit is configured to generate aplurality of different initial groupings of the image slices and displaya plurality of slabbed views of the initial groupings.
 5. A systemaccording to claim 4, wherein the number of slices in a respectiveinitial grouping of slices is less where there is at least one CADidentified suspected abnormality relative to the number of slices ininitial slice groupings that do have not have at least one CADidentified suspected abnormality.
 6. A system according to claim 1,wherein the at least one grouping is a plurality of groupings of slices,wherein the processor circuit is configured to generate a plurality ofslabbed views using different groupings of slices, and wherein the GUIis configured to allow a user to browse the different slabbed views. 7.A system according to claim 6, wherein the GUI is configured to allow auser to refine a slabbed view whereby the current group of slicesassociated with that slabbed view is split into a plurality of separatesmaller groups of slices and/or iteratively into a single slice view. 8.A system according to claim 1, wherein the circuit is configured toprovide an electronic review wizard with an automated viewing protocolthat allows a user to browse the slabbed views.
 9. A system according toclaim 8, wherein the wizard is configured to display slabbed views ofinitial groupings and allow a user to select refinements of one or moreof the initial groupings whereby a selected slabbed view is split into aplurality of separate smaller groups of slices and/or into a singleslice view.
 10. A system according to claim 8, wherein the wizard isconfigured to display the slabbed views of the initial groupings andautomatically show refinements of the initial groupings whereby thecurrent slabbed view being refined is split into a plurality of separategroups of slices and/or into a single slice view.
 11. A system accordingto claim 1, wherein the at least one slabbed view is generated by takingthe average of the pixel values at a corresponding position in differentslices of a respective grouping.
 12. A system according to claim 1,wherein the at least one slabbed view is generated by taking the maximumof the pixel values at a corresponding position in different slices of arespective grouping.
 13. A system according to claim 1, wherein the atleast one slabbed view is generated by taking the minimum of the pixelvalues at a corresponding position in different slices of a respectivegrouping.
 14. A system according to claim 1, wherein the at least oneslabbed view is generated by taking the median of the pixel values at acorresponding position in different slices of a respective grouping. 15.A method of generating views of medical data, comprising: electronicallygenerating groups of slices from a stack of slices using Computer AidedDetection (CAD) to electronically analyze and identify suspectedabnormalities in slices of the medical image data stack andautomatically generate the groups of slices based on the CADidentification of abnormalities; combining pixel values associated witha common position in each slice of a respective group; and generatingslabbed views, one for each of the respective grouped slices, based onthe combined pixel values.
 16. A method according to claim 15, furthercomprising refining at least one of the slabbed views into differentsub-groupings of the original group of slices for the refined slabbedview.
 17. A method according to claim 15, further comprising iterativelyrefining at least one of the slabbed views into smaller sub-groupings ofslices and/or into single slices; then generating associated refinedslabbed views.
 18. A method according to claim 15, further comprisingaccepting user input to select to browse the slabbed views and to refinethe slabbed views.
 19. A processor circuit for generating medicaldiagnostic views of a tomosynthesis stack of mammography patient imagedata, wherein the circuit is configured to communicate with a graphicaluser interface (GUI) associated with a client workstation to accept userinput to interact with the image data to generate desired views of theimage data, and wherein the circuit is configured to employ ComputerAided Detection (CAD) to electronically analyze and identify suspectedabnormalities in slices of the tomosynthesis image data stack to definegroups of slices and to generate slabbed views of breast tissue withsuspected abnormalities and allow a user to browse the slabbed views.20. A computer program product for providing physician interactiveaccess to patient medical data for rendering diagnostic medical images,the computer program product comprising: a non-transitory computerreadable storage medium having computer readable program code embodiedin the medium, the computer-readable program code comprising: computerreadable program code configured to employ Computer Aided Detection(CAD) to electronically analyze and identify suspected abnormalities inslices of a stack of patient medical image data to define groups ofslices of a medical image data stack; and computer readable program codeconfigured to generate a respective slabbed view for each of the groupsof slices with suspected abnormalities.